Skip to content

[ty] Respect type variables in top/bottom materializations of invariant generics - #28072

Draft
sharkdp wants to merge 11 commits into
mainfrom
codex/invariant-typevar-materializations
Draft

[ty] Respect type variables in top/bottom materializations of invariant generics#28072
sharkdp wants to merge 11 commits into
mainfrom
codex/invariant-typevar-materializations

Conversation

@sharkdp

@sharkdp sharkdp commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Summary

ty could ignore an inferable type variable when comparing top materializations of generic types. For example, it considered Top[list[Unknown]] to be a subtype of list[T].

Here, we remove this TODO/shortcut, so union simplification and narrowing preserve all possible alternatives. Related overlap checks and callable-parameter (ParamSpec) comparisons are adjusted to work without it.

The extra check discussed in this review comment is now unnecessary and is removed.

Related previous PR: Preserve invariant materialization constraints.

closes astral-sh/ty#4201

Ecosystem

Looks like those are new true positives in strict-mode projects.

"The stable changes are expected consequences of fixing invariant comparisons involving type variables: a top materialization such as Top[list[Unknown]] no longer incorrectly satisfies the fixed specialization list[T], and valid union alternatives are no longer discarded on that basis. All additions occur in projects checked with strict generic narrowing. Archinstall’s non-strict result removes an incorrect redundant-cast warning. Prefect has one secondary diagnostic replacement caused by recovery from a newly rejected assignment; it does not establish a new runtime None path.

The comparison adds 21 stable diagnostics and removes two, including the Prefect replacement. I found no additional semantic regression in these changes."

Test plan

New regression tests

@sharkdp sharkdp added the ty Multi-file analysis & type inference label Aug 26, 2026
@sharkdp
sharkdp force-pushed the codex/invariant-typevar-materializations branch from e8d39af to 55aa8f4 Compare August 27, 2026 07:03
@astral-sh-bot

astral-sh-bot Bot commented Aug 27, 2026

Copy link
Copy Markdown

Typing conformance results

No changes detected ✅

Current numbers
The percentage of diagnostics emitted that were expected errors held steady at 97.79%. The percentage of expected errors that received a diagnostic held steady at 94.33%. The number of fully passing files held steady at 112/136.

@astral-sh-bot

astral-sh-bot Bot commented Aug 27, 2026

Copy link
Copy Markdown

Memory usage report

Summary

Project Old New Diff Outcome
prefect 477.73MB 477.79MB +0.01% (57.40kB)
trio 96.96MB 96.98MB +0.02% (20.29kB)
sphinx 172.97MB 172.97MB +0.00% (1.61kB)
flake8 40.23MB 40.23MB -

Significant changes

Click to expand detailed breakdown

prefect

Name Old New Diff Outcome
infer_expression_types_impl 24.43MB 24.40MB -0.10% (24.86kB) ⬇️
Specialization 5.01MB 5.02MB +0.24% (12.48kB)
when_constraint_set_assignable_to_owned_impl 3.92MB 3.93MB +0.31% (12.45kB)
TypePair 8.06MB 8.07MB +0.12% (10.31kB)
TypeVarSetInner 1.12MB 1.13MB +0.79% (9.04kB)
Type<'db>::cached_materialization_::interned_arguments 1.50MB 1.50MB +0.47% (7.27kB)
StaticClassLiteral<'db>::try_mro_specialized_ 4.49MB 4.50MB +0.10% (4.66kB)
is_redundant_with_impl 3.44MB 3.45MB +0.12% (4.38kB)
infer_definition_types 37.06MB 37.07MB +0.01% (4.24kB)
Type<'db>::cached_materialization_ 1.14MB 1.14MB +0.34% (3.98kB)
BoundTypeVarInstance 1.61MB 1.61MB +0.22% (3.61kB)
union_from_two_elements 177.03kB 180.40kB +1.90% (3.37kB)
ClassType<'db>::nearest_disjoint_base_ 262.69kB 265.23kB +0.97% (2.55kB)
is_possibly_constraint_set_assignable 346.73kB 348.80kB +0.60% (2.07kB)
Type<'db>::apply_specialization_inner_::interned_arguments 5.42MB 5.42MB -0.03% (1.80kB) ⬇️
... 50 more

trio

Name Old New Diff Outcome
infer_expression_types_impl 6.42MB 6.42MB +0.03% (2.27kB)
merged_specialization_inner 134.10kB 136.36kB +1.68% (2.26kB)
TypeVarSetInner 263.66kB 265.32kB +0.63% (1.66kB)
when_constraint_set_assignable_to_owned_impl 747.42kB 748.83kB +0.19% (1.41kB)
Type<'db>::apply_specialization_inner_::interned_arguments 1.26MB 1.26MB +0.10% (1.33kB)
FunctionType 1.43MB 1.43MB +0.07% (1.05kB)
Type<'db>::apply_specialization_inner_ 879.32kB 880.30kB +0.11% (1000.00B)
TypePair 1.04MB 1.04MB +0.09% (960.00B)
Specialization 1.12MB 1.12MB +0.08% (896.00B)
Type<'db>::class_member_with_policy_inner_ 1.29MB 1.29MB +0.06% (800.00B)
GenericContext 571.44kB 572.11kB +0.12% (684.00B)
infer_definition_types 4.73MB 4.73MB +0.01% (600.00B)
MemberLookupKey 1.14MB 1.14MB +0.04% (520.00B)
TypeVarInference 344.71kB 345.18kB +0.14% (480.00B)
member_lookup_with_policy_inner 1.25MB 1.25MB +0.03% (440.00B)
... 31 more

sphinx

Name Old New Diff Outcome
when_constraint_set_assignable_to_owned_impl 908.68kB 909.09kB +0.05% (424.00B)
TypePair 2.17MB 2.17MB +0.02% (384.00B)
infer_expression_types_impl 11.95MB 11.95MB +0.00% (312.00B)
is_redundant_with_impl 790.74kB 791.00kB +0.03% (264.00B)
infer_definition_types 11.59MB 11.59MB +0.00% (264.00B)
Type<'db>::apply_specialization_inner_::interned_arguments 1.83MB 1.82MB -0.01% (240.00B) ⬇️
TypeVarSetInner 452.88kB 453.02kB +0.03% (148.00B)
union_from_two_elements 76.46kB 76.60kB +0.18% (144.00B)
analyze_non_terminal_call 483.15kB 483.29kB +0.03% (144.00B)
Type<'db>::apply_specialization_inner_ 1.16MB 1.16MB -0.01% (144.00B) ⬇️
TypeVarInference 447.98kB 447.86kB -0.03% (120.00B) ⬇️
all_narrowing_constraints_for_expression 2.39MB 2.39MB +0.00% (96.00B)
loop_header_reachability 250.14kB 250.24kB +0.04% (96.00B)
GenericAlias 663.89kB 663.82kB -0.01% (72.00B) ⬇️
is_possibly_constraint_set_assignable 84.78kB 84.74kB -0.05% (40.00B) ⬇️
... 2 more

@astral-sh-bot

astral-sh-bot Bot commented Aug 27, 2026

Copy link
Copy Markdown

ecosystem-analyzer results

Lint rule Added Removed Changed
invalid-argument-type 18 0 12
invalid-return-type 8 0 6
invalid-assignment 4 0 9
invalid-yield 3 0 0
unsound-return-statement 2 1 0
no-matching-overload 1 1 0
type-assertion-failure 2 0 0
redundant-cast 0 1 0
unresolved-attribute 1 0 0
Total 39 3 27
Raw diff (69 changes)
anyio (https://github.com/agronholm/anyio)
+ src/anyio/itertools.py:116:13 error[invalid-assignment] Object of type `Top[_TeeState[Unknown]]` is not assignable to attribute `_state` of type `_TeeState[T@_TeeAsyncIterator]`
+ src/anyio/itertools.py:117:13 error[invalid-assignment] Object of type `Top[_TeeLink[Unknown]]` is not assignable to attribute `_link` of type `_TeeLink[T@_TeeAsyncIterator]`

archinstall (https://github.com/archlinux/archinstall)
- archinstall/tui/result.py:71:10 warning[redundant-cast] Value is already of type `list[ValueT@Result]`

cki-lib (https://gitlab.com/cki-project/cki-lib)
+ cki_lib/misc.py:68:24 error[invalid-yield] Yield type `object` does not match annotated yield type `T@flattened`

core (https://github.com/home-assistant/core)
+ homeassistant/components/datadog/__init__.py:85:31 error[invalid-argument-type] Argument to bound method `EventBus.async_listen` is incorrect: Expected `EventType[Mapping[str, Any]] | str`, found `EventType[EventStateChangedData]`
+ homeassistant/components/influxdb/__init__.py:595:35 error[invalid-argument-type] Argument to bound method `EventBus.async_listen` is incorrect: Expected `EventType[Mapping[str, Any]] | str`, found `EventType[EventStateChangedData]`
+ homeassistant/components/statsd/__init__.py:91:21 error[invalid-argument-type] Argument to bound method `EventBus.listen` is incorrect: Expected `EventType[Mapping[str, Any]] | str`, found `EventType[EventStateChangedData]`
- homeassistant/core.py:2482:17 error[invalid-argument-type] Argument to bound method `EventBus.async_fire_internal` is incorrect: Expected `EventType[EventStateReportedData | dict[str, str | datetime | State | None]] | str`, found `EventType[EventStateReportedData]`
+ homeassistant/core.py:2482:17 error[invalid-argument-type] Argument to bound method `EventBus.async_fire_internal` is incorrect: Expected `EventType[dict[str, str | datetime | State | None]] | str`, found `EventType[EventStateReportedData]`
+ homeassistant/helpers/device_registry.py:2941:13 error[invalid-argument-type] Argument to bound method `EventBus.async_fire_internal` is incorrect: Expected `EventType[_EventDeviceRegistryUpdatedData_Update] | str`, found `EventType[EventDeviceRegistryUpdatedData]`
+ homeassistant/helpers/device_registry.py:3432:43 error[invalid-argument-type] Argument to bound method `EventBus.async_fire_internal` is incorrect: Expected `EventType[_EventDeviceRegistryUpdatedData_Create | _EventDeviceRegistryUpdatedData_Update] | str`, found `EventType[EventDeviceRegistryUpdatedData]`
+ homeassistant/helpers/device_registry.py:3615:43 error[invalid-argument-type] Argument to bound method `EventBus.async_fire_internal` is incorrect: Expected `EventType[_EventDeviceRegistryUpdatedData_Create | _EventDeviceRegistryUpdatedData_Update] | str`, found `EventType[EventDeviceRegistryUpdatedData]`
+ homeassistant/helpers/device_registry.py:4104:13 error[invalid-argument-type] Argument to bound method `EventBus.async_fire_internal` is incorrect: Expected `EventType[_EventDeviceRegistryUpdatedData_Remove] | str`, found `EventType[EventDeviceRegistryUpdatedData]`
+ homeassistant/helpers/device_registry.py:4135:13 error[invalid-argument-type] Argument to bound method `EventBus.async_fire_internal` is incorrect: Expected `EventType[_EventDeviceRegistryUpdatedData_Remove] | str`, found `EventType[EventDeviceRegistryUpdatedData]`
+ homeassistant/helpers/entity_registry.py:1616:13 error[invalid-argument-type] Argument to bound method `EventBus.async_fire_internal` is incorrect: Expected `EventType[_EventEntityRegistryUpdatedData_CreateRemove] | str`, found `EventType[EventEntityRegistryUpdatedData]`
+ homeassistant/helpers/entity_registry.py:1664:13 error[invalid-argument-type] Argument to bound method `EventBus.async_fire_internal` is incorrect: Expected `EventType[_EventEntityRegistryUpdatedData_CreateRemove] | str`, found `EventType[EventEntityRegistryUpdatedData]`
+ homeassistant/helpers/entity_registry.py:2014:43 error[invalid-argument-type] Argument to bound method `EventBus.async_fire_internal` is incorrect: Expected `EventType[_EventEntityRegistryUpdatedData_Update] | str`, found `EventType[EventEntityRegistryUpdatedData]`
+ homeassistant/helpers/floor_registry.py:225:13 error[invalid-argument-type] Argument to bound method `EventBus.async_fire_internal` is incorrect: Expected `EventType[_EventFloorRegistryUpdatedData_Create_Remove_Update] | str`, found `EventType[EventFloorRegistryUpdatedData]`
+ homeassistant/helpers/floor_registry.py:238:13 error[invalid-argument-type] Argument to bound method `EventBus.async_fire_internal` is incorrect: Expected `EventType[_EventFloorRegistryUpdatedData_Create_Remove_Update] | str`, found `EventType[EventFloorRegistryUpdatedData]`
+ homeassistant/helpers/floor_registry.py:280:13 error[invalid-argument-type] Argument to bound method `EventBus.async_fire_internal` is incorrect: Expected `EventType[_EventFloorRegistryUpdatedData_Create_Remove_Update] | str`, found `EventType[EventFloorRegistryUpdatedData]`
+ homeassistant/helpers/floor_registry.py:307:13 error[invalid-argument-type] Argument to bound method `EventBus.async_fire_internal` is incorrect: Expected `EventType[_EventFloorRegistryUpdatedData_Reorder] | str`, found `EventType[EventFloorRegistryUpdatedData]`
- homeassistant/scripts/benchmark/__init__.py:143:29 error[invalid-argument-type] Argument to bound method `EventBus.async_fire` is incorrect: Expected `EventType[EventStateChangedData | dict[str, str | State]] | str`, found `EventType[EventStateChangedData]`
+ homeassistant/scripts/benchmark/__init__.py:143:29 error[invalid-argument-type] Argument to bound method `EventBus.async_fire` is incorrect: Expected `EventType[dict[str, str | State]] | str`, found `EventType[EventStateChangedData]`
- homeassistant/scripts/benchmark/__init__.py:176:29 error[invalid-argument-type] Argument to bound method `EventBus.async_fire` is incorrect: Expected `EventType[EventStateChangedData | dict[str, str | State]] | str`, found `EventType[EventStateChangedData]`
+ homeassistant/scripts/benchmark/__init__.py:176:29 error[invalid-argument-type] Argument to bound method `EventBus.async_fire` is incorrect: Expected `EventType[dict[str, str | State]] | str`, found `EventType[EventStateChangedData]`
- homeassistant/scripts/benchmark/__init__.py:214:29 error[invalid-argument-type] Argument to bound method `EventBus.async_fire` is incorrect: Expected `EventType[EventStateChangedData | dict[str, str | State]] | str`, found `EventType[EventStateChangedData]`
+ homeassistant/scripts/benchmark/__init__.py:214:29 error[invalid-argument-type] Argument to bound method `EventBus.async_fire` is incorrect: Expected `EventType[dict[str, str | State]] | str`, found `EventType[EventStateChangedData]`

discord.py (https://github.com/Rapptz/discord.py)
+ discord/ext/commands/cog.py:461:32 error[invalid-yield] Yield type `Top[Command[Unknown, (...), Any]]` does not match annotated yield type `Command[Self@walk_commands, (...), Any]`
+ discord/ext/commands/core.py:1455:28 error[invalid-yield] Yield type `Top[Command[Unknown, (...), Any]]` does not match annotated yield type `Command[CogT@GroupMixin, (...), Any]`
+ discord/ext/commands/core.py:1499:16 error[invalid-return-type] Return type does not match returned value: expected `Command[CogT@GroupMixin, (...), Any] | None`, found `Top[Command[Unknown, (...), Any]] | Unknown`

meson (https://github.com/mesonbuild/meson)
+ mesonbuild/modules/hotdoc.py:61:12 error[invalid-return-type] Return type does not match returned value: expected `list[_T@ensure_list]`, found `(_T@ensure_list & Top[list[Unknown]]) | list[_T@ensure_list]`
+ mesonbuild/utils/universal.py:1912:12 error[invalid-return-type] Return type does not match returned value: expected `list[_T@typeslistify]`, found `(_T@typeslistify & Top[list[Unknown]]) | Top[list[Unknown & _T@typeslistify]]`

mkdocs (https://github.com/mkdocs/mkdocs)
+ mkdocs/config/config_options.py:219:20 error[invalid-return-type] Return type does not match returned value: expected `list[T@ListOfItems]`, found `Top[list[Unknown]] & ~AlwaysTruthy`

optuna (https://github.com/optuna/optuna)
- optuna/storages/_rdb/alembic/versions/v1.3.0.a.py:63:59 error[invalid-argument-type] Argument to function `loads` is incorrect: Expected `str | bytes | bytearray`, found `Column[str]`
+ optuna/storages/_rdb/alembic/versions/v1.3.0.a.py:63:59 error[invalid-argument-type] Argument to function `loads` is incorrect: Expected `str | bytes | bytearray`, found `Column[Unknown]`
- optuna/storages/_rdb/alembic/versions/v3.0.0.a.py:158:13 error[invalid-assignment] Object of type `str` is not assignable to attribute `distribution_json` of type `Column[str]`
+ optuna/storages/_rdb/alembic/versions/v3.0.0.a.py:158:13 error[invalid-assignment] Object of type `str` is not assignable to attribute `distribution_json` of type `Column[Unknown]`
- optuna/storages/_rdb/alembic/versions/v3.0.0.a.py:159:17 error[invalid-argument-type] Argument to function `migrate_new_distribution` is incorrect: Expected `str`, found `Column[str]`
+ optuna/storages/_rdb/alembic/versions/v3.0.0.a.py:159:17 error[invalid-argument-type] Argument to function `migrate_new_distribution` is incorrect: Expected `str`, found `Column[Unknown]`
- optuna/storages/_rdb/alembic/versions/v3.0.0.a.py:187:13 error[invalid-assignment] Object of type `str` is not assignable to attribute `distribution_json` of type `Column[str]`
+ optuna/storages/_rdb/alembic/versions/v3.0.0.a.py:187:13 error[invalid-assignment] Object of type `str` is not assignable to attribute `distribution_json` of type `Column[Unknown]`
- optuna/storages/_rdb/alembic/versions/v3.0.0.a.py:188:17 error[invalid-argument-type] Argument to function `restore_old_distribution` is incorrect: Expected `str`, found `Column[str]`
+ optuna/storages/_rdb/alembic/versions/v3.0.0.a.py:188:17 error[invalid-argument-type] Argument to function `restore_old_distribution` is incorrect: Expected `str`, found `Column[Unknown]`
- optuna/storages/_rdb/alembic/versions/v3.0.0.c.py:128:25 error[invalid-assignment] Object of type `Column[float]` is not assignable to `float`
+ optuna/storages/_rdb/alembic/versions/v3.0.0.c.py:128:25 error[invalid-assignment] Object of type `Column[Unknown]` is not assignable to `float`
- optuna/storages/_rdb/alembic/versions/v3.0.0.d.py:132:25 error[invalid-assignment] Object of type `Column[float]` is not assignable to `float`
+ optuna/storages/_rdb/alembic/versions/v3.0.0.d.py:132:25 error[invalid-assignment] Object of type `Column[Unknown]` is not assignable to `float`
- optuna/storages/_rdb/storage.py:342:17 error[invalid-assignment] Invalid assignment to data descriptor attribute `value_json` on type `StudyUserAttributeModel`: Argument type `MappedColumn[Any] | Column[str]` does not satisfy upper bound `Mapped[Any]` of type variable `Self`
+ optuna/storages/_rdb/storage.py:342:17 error[invalid-assignment] Invalid assignment to data descriptor attribute `value_json` on type `StudyUserAttributeModel`: Argument type `MappedColumn[Any] | Column[Unknown]` does not satisfy upper bound `Mapped[Any]` of type variable `Self`
- optuna/storages/_rdb/storage.py:354:17 error[invalid-assignment] Invalid assignment to data descriptor attribute `value_json` on type `StudySystemAttributeModel`: Argument type `MappedColumn[Any] | Column[str]` does not satisfy upper bound `Mapped[Any]` of type variable `Self`
+ optuna/storages/_rdb/storage.py:354:17 error[invalid-assignment] Invalid assignment to data descriptor attribute `value_json` on type `StudySystemAttributeModel`: Argument type `MappedColumn[Any] | Column[Unknown]` does not satisfy upper bound `Mapped[Any]` of type variable `Self`
- optuna/storages/_rdb/storage.py:368:16 error[invalid-return-type] Return type does not match returned value: expected `str`, found `Any | Column[str]`
+ optuna/storages/_rdb/storage.py:368:16 error[invalid-return-type] Return type does not match returned value: expected `str`, found `Any | Column[Unknown]`
- optuna/storages/_rdb/storage.py:382:48 error[invalid-argument-type] Argument to function `loads` is incorrect: Expected `str | bytes | bytearray`, found `Any | Column[str]`
+ optuna/storages/_rdb/storage.py:382:48 error[invalid-argument-type] Argument to function `loads` is incorrect: Expected `str | bytes | bytearray`, found `Any | Column[Unknown]`
- optuna/storages/_rdb/storage.py:384:16 error[invalid-return-type] Return type does not match returned value: expected `dict[str, Any]`, found `dict[Any | Column[str], Any]`
+ optuna/storages/_rdb/storage.py:384:16 error[invalid-return-type] Return type does not match returned value: expected `dict[str, Any]`, found `dict[Any | Column[Unknown], Any]`
- optuna/storages/_rdb/storage.py:391:50 error[invalid-argument-type] Argument to function `loads` is incorrect: Expected `str | bytes | bytearray`, found `Any | Column[str]`
+ optuna/storages/_rdb/storage.py:391:50 error[invalid-argument-type] Argument to function `loads` is incorrect: Expected `str | bytes | bytearray`, found `Any | Column[Unknown]`
- optuna/storages/_rdb/storage.py:393:16 error[invalid-return-type] Return type does not match returned value: expected `dict[str, Any]`, found `dict[Any | Column[str], Any]`
+ optuna/storages/_rdb/storage.py:393:16 error[invalid-return-type] Return type does not match returned value: expected `dict[str, Any]`, found `dict[Any | Column[Unknown], Any]`
- optuna/storages/_rdb/storage.py:401:48 error[invalid-argument-type] Argument to function `loads` is incorrect: Expected `str | bytes | bytearray`, found `Any | Column[str]`
+ optuna/storages/_rdb/storage.py:401:48 error[invalid-argument-type] Argument to function `loads` is incorrect: Expected `str | bytes | bytearray`, found `Any | Column[Unknown]`
- optuna/storages/_rdb/storage.py:403:16 error[invalid-return-type] Return type does not match returned value: expected `dict[str, Any]`, found `dict[Any | Column[str], Any]`
+ optuna/storages/_rdb/storage.py:403:16 error[invalid-return-type] Return type does not match returned value: expected `dict[str, Any]`, found `dict[Any | Column[Unknown], Any]`
- optuna/storages/_rdb/storage.py:411:50 error[invalid-argument-type] Argument to function `loads` is incorrect: Expected `str | bytes | bytearray`, found `Any | Column[str]`
+ optuna/storages/_rdb/storage.py:411:50 error[invalid-argument-type] Argument to function `loads` is incorrect: Expected `str | bytes | bytearray`, found `Any | Column[Unknown]`
- optuna/storages/_rdb/storage.py:413:16 error[invalid-return-type] Return type does not match returned value: expected `dict[str, Any]`, found `dict[Any | Column[str], Any]`
+ optuna/storages/_rdb/storage.py:413:16 error[invalid-return-type] Return type does not match returned value: expected `dict[str, Any]`, found `dict[Any | Column[Unknown], Any]`
- optuna/storages/_rdb/storage.py:645:21 error[invalid-argument-type] Argument to function `json_to_distribution` is incorrect: Expected `str`, found `Any | Column[str]`
+ optuna/storages/_rdb/storage.py:645:21 error[invalid-argument-type] Argument to function `json_to_distribution` is incorrect: Expected `str`, found `Any | Column[Unknown]`
- optuna/storages/_rdb/storage.py:668:16 error[invalid-return-type] Return type does not match returned value: expected `float`, found `Any | Column[float]`
+ optuna/storages/_rdb/storage.py:668:16 error[invalid-return-type] Return type does not match returned value: expected `float`, found `Any | Column[Unknown]`
- optuna/storages/_rdb/storage.py:716:13 error[invalid-assignment] Invalid assignment to data descriptor attribute `value` on type `TrialValueModel`: Argument type `MappedColumn[Any] | Column[float]` does not satisfy upper bound `Mapped[Any]` of type variable `Self`
+ optuna/storages/_rdb/storage.py:716:13 error[invalid-assignment] Invalid assignment to data descriptor attribute `value` on type `TrialValueModel`: Argument type `MappedColumn[Any] | Column[Unknown]` does not satisfy upper bound `Mapped[Any]` of type variable `Self`
- optuna/storages/_rdb/storage.py:755:13 error[invalid-assignment] Invalid assignment to data descriptor attribute `intermediate_value` on type `TrialIntermediateValueModel`: Argument type `MappedColumn[Any] | Column[float]` does not satisfy upper bound `Mapped[Any]` of type variable `Self`
+ optuna/storages/_rdb/storage.py:755:13 error[invalid-assignment] Invalid assignment to data descriptor attribute `intermediate_value` on type `TrialIntermediateValueModel`: Argument type `MappedColumn[Any] | Column[Unknown]` does not satisfy upper bound `Mapped[Any]` of type variable `Self`
- optuna/storages/_rdb/storage.py:1211:13 error[invalid-assignment] Invalid assignment to data descriptor attribute `library_version` on type `VersionInfoModel`: Argument type `MappedColumn[Any] | Column[str]` does not satisfy upper bound `Mapped[Any]` of type variable `Self`
+ optuna/storages/_rdb/storage.py:1211:13 error[invalid-assignment] Invalid assignment to data descriptor attribute `library_version` on type `VersionInfoModel`: Argument type `MappedColumn[Any] | Column[Unknown]` does not satisfy upper bound `Mapped[Any]` of type variable `Self`

pandas-stubs (https://github.com/pandas-dev/pandas-stubs)
+ tests/test_pandas.py:525:11 error[type-assertion-failure] Type `ndarray[tuple[int], dtype[numpy.bool[builtins.bool]]]` does not match asserted type `ndarray[tuple[Any, ...], dtype[numpy.bool[builtins.bool]]]`
+ tests/test_pandas.py:526:11 error[type-assertion-failure] Type `ndarray[tuple[int], dtype[numpy.bool[builtins.bool]]]` does not match asserted type `ndarray[tuple[Any, ...], dtype[numpy.bool[builtins.bool]]]`

prefect (https://github.com/PrefectHQ/prefect)
- src/integrations/prefect-redis/tests/test_tasks.py:52:12 warning[unsound-return-statement] Unsound return statement: `Unknown` is not a subtype of `str`
+ src/prefect/_internal/states.py:239:16 error[invalid-return-type] Return type does not match returned value: expected `State[R@return_value_to_state_sync]`, found `R@return_value_to_state_sync & Top[State[Unknown]]`
+ src/prefect/futures.py:252:13 error[invalid-assignment] Object of type `R@PrefectConcurrentFuture & Top[State[Unknown]]` is not assignable to attribute `_final_state` of type `State[R@PrefectConcurrentFuture] | None`
+ src/prefect/futures.py:270:17 error[invalid-assignment] Object of type `R@PrefectConcurrentFuture & Top[State[Unknown]]` is not assignable to attribute `_final_state` of type `State[R@PrefectConcurrentFuture] | None`
- src/prefect/futures.py:277:19 error[no-matching-overload] No overload of bound method `State.result` matches arguments
+ src/prefect/futures.py:277:19 error[unresolved-attribute] Attribute `result` is not defined on `None` in union `State[R@PrefectConcurrentFuture] | None`
+ src/prefect/input/run_input.py:890:16 error[invalid-return-type] Return type does not match returned value: expected `GetAutomaticInputHandler[T@receive_input] | GetInputHandler[R@receive_input]`, found `Top[GetAutomaticInputHandler[Unknown]]`
+ src/prefect/server/database/query_components.py:577:13 warning[unsound-return-statement] Unsound return statement: `Select[tuple[Unknown, datetime]]` is not a subtype of `Select[tuple[datetime, datetime]]`
+ src/prefect/states.py:369:16 error[invalid-return-type] Return type does not match returned value: expected `State[R@return_value_to_state]`, found `R@return_value_to_state & Top[State[Unknown]]`
+ src/prefect/task_engine.py:628:34 error[invalid-argument-type] Argument to bound method `SyncTaskRunEngine.handle_retry` is incorrect: Expected `Exception | State[R@SyncTaskRunEngine]`, found `R@SyncTaskRunEngine & Top[State[Unknown]]`
+ src/prefect/task_engine.py:1253:40 error[invalid-argument-type] Argument to bound method `AsyncTaskRunEngine.handle_retry` is incorrect: Expected `Exception | State[R@AsyncTaskRunEngine]`, found `R@AsyncTaskRunEngine & Top[State[Unknown]]`

scrapy (https://github.com/scrapy/scrapy)
+ scrapy/utils/defer.py:392:16 error[invalid-return-type] Return type does not match returned value: expected `Deferred[_T@deferred_from_coro] | _T2@deferred_from_coro`, found `(Awaitable[_T@deferred_from_coro] & Top[Deferred[Unknown]]) | (_T2@deferred_from_coro & Top[Deferred[Unknown]])`
+ scrapy/utils/defer.py:571:41 error[invalid-argument-type] Argument to function `maybe_deferred_to_future` is incorrect: Expected `Deferred[_T@ensure_awaitable]`, found `(_T@ensure_awaitable & Top[Deferred[Unknown]]) | (Awaitable[_T@ensure_awaitable] & Top[Deferred[Unknown]])`

streamlit (https://github.com/streamlit/streamlit)
+ lib/streamlit/runtime/caching/cache_data_api.py:831:20 warning[unsound-return-statement] Unsound return statement: `Any & Top[CachedResult[Unknown]]` is not a subtype of `CachedResult[R@DataCache]`

tornado (https://github.com/tornadoweb/tornado)
+ tornado/concurrent.py:170:9 error[no-matching-overload] No overload of function `future_add_done_callback` matches arguments

Full report with detailed diff (timing results)

@codspeed-hq

codspeed-hq Bot commented Aug 27, 2026

Copy link
Copy Markdown

Merging this PR will degrade performance by 23.28%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

❌ 1 regressed benchmark
✅ 125 untouched benchmarks
⏩ 84 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation ty_micro[invariant_generic_return_union] 198.7 ms 259 ms -23.28%

Tip

Investigate this regression with the CodSpeed MCP and your agent.


Comparing codex/invariant-typevar-materializations (88059dd) with main (c0b78da)

Open in CodSpeed

Footnotes

  1. 84 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

Validate possible overlap against TypeVar bounds and constraints, while retaining incompatible union alternatives so inference can still report argument errors. Add coverage for bounded and constrained nested type variables.

Compare fixed ParamSpecs directly with materialization endpoints and clarify the parameter kind distinction between top and bottom.
from typing import assert_never
from typing import assert_never, final

@final

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without @final, we would now reveal Box[T@box_with_default] | (T@box_with_default & Top[Box[Unknown]]) in the first isinstance branch below. I believe this is correct (and was wrong on main): it accounts for the possibility of a common subclass of str and Box (possibly with another specialization). The return value consequently lead to an error.

I added @final to restore the original intention of this test (similar in match.md).

Comment on lines +1218 to +1220
static_assert(not is_disjoint_from(list[list[T]], list[list[int]]))
static_assert(not is_disjoint_from(list[list[Id[T]]], list[list[int]]))
static_assert(not is_disjoint_from(list[list[T]], list[list[Never]]))

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these fail on main

Comment on lines +1230 to +1231
static_assert(not is_disjoint_from(list[list[T]], list[list[str]]))
static_assert(not is_disjoint_from(list[list[T]], list[list[Never]]))

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these fail on main

Comment on lines +1242 to +1243
static_assert(not is_disjoint_from(list[list[T]], list[list[str]]))
static_assert(not is_disjoint_from(list[list[T]], list[list[bytes]]))

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these fail on main

Comment on lines +304 to +305
static_assert(not is_subtype_of(Top[Box[...]], Box[P]))
static_assert(not is_subtype_of(Box[P], Bottom[Box[...]]))

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these two fail on main

Comment on lines +309 to +310
static_assert(not is_assignable_to(Top[Box[...]], Box[P]))
static_assert(not is_assignable_to(Box[P], Bottom[Box[...]]))

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these two fail on main

@sharkdp sharkdp changed the title [ty] Preserve type variables in invariant materialization subtyping [ty] Respect type variables in top/bottom materializations of invariant generics Aug 28, 2026
Comment on lines +520 to +551
/// Returns whether no specialization satisfying the type variables' upper bounds and
/// constraints can satisfy this constraint set.
///
/// Unlike [`Self::is_never_satisfied`], this validates solutions against the type variables'
/// upper bounds and constraints. For example, `T = int` is not contradictory by itself, but has
/// no valid solution if `T` has an upper bound of `str`.
///
/// If the solver reaches its computation limit, we do not know whether a valid solution exists.
/// This returns `false` in that case: stopping the search is not proof that there is no solution.
pub(crate) fn has_no_valid_solutions(
self,
db: &'db dyn Db,
env: &ProgramEnvironment<'db>,
) -> bool {
if self.is_never_satisfied(db, env) {
return true;
}

let inferable = {
let storage = self.builder.storage.borrow();
let Some(support) = storage.node_support(self.node) else {
return false;
};
// For overlap, every mentioned type variable can choose a valid specialization.
TypeVarSet::from_typevars(db, support.iter().map(|id| storage.typevar_data(id)))
};

matches!(
self.solutions(db, env, inferable),
Ok(Solutions::Unsatisfiable)
)
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe if this TODO get's resolved, we won't need this function?

@sharkdp
sharkdp marked this pull request as ready for review August 28, 2026 16:10
@sharkdp
sharkdp requested a review from a team as a code owner August 28, 2026 16:10
@astral-sh-bot
astral-sh-bot Bot requested a review from carljm August 28, 2026 16:10
@sharkdp

sharkdp commented Aug 28, 2026

Copy link
Copy Markdown
Contributor Author

Hm, the latest ecosystem analysis found two cases (below) that should be fixed before merging. I should also check if that performance regression is real.

Details

Both examples reproduce without strict-analysis flags.

  1. A gradual bound causes valid type information to be lost:

    from typing import Any, TypeVar, reveal_type
    
    T = TypeVar("T", bound=Any)
    
    class Other:
        pass
    
    def infer(value: list[T] | Other) -> T:
        raise NotImplementedError
    
    reveal_type(infer(list[str]()))
    # Base: str
    # PR:   Unknown

    T = str is valid, but the PR incorrectly discards the generic union arm during inference.

  2. Partial union filtering allows an argument that violates the bound:

    from typing import TypeVar
    
    T = TypeVar("T", bound=str)
    
    class Other:
        pass
    
    def accept(value: list[T] | Other):
        pass
    
    accept([object()])
    # Base: invalid-argument-type — object does not satisfy upper bound str
    # PR:   accepted

    Here, discarding list[T] loses its bound check while Other survives.

@sharkdp
sharkdp marked this pull request as draft August 28, 2026 17:37
@carljm

carljm commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

The micro-benchmark seems suspiciously related, might be real -- and I might accept that regression in a micro-benchmark if it's in exchange for fixing this terrible TODO :) but worth seeing if we can find a way to reduce.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ty Multi-file analysis & type inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Top[list[Unknown]] incorrectly considered a subtype of list[T]

2 participants